Recent Results in Controlled Markov Chains with Risk Sensitive Average Criteria: the Vanishing Discount Approach

Countable state space Markov cost/ reward chains, satisfying a Lyapunov-t ype stability condition, are considered in this work. For an infinite planning horizon, risk sensitive (exponential) discounted and average cost criteria are considered. The main contribution is the development of a vanishing discount approach to relate the discounted criterion problem with the average criterion one, as the discount factor increases to one, i.e., no discounting. In comparison to the well-established risk–neutral case, our results are novel and reveal several fundamental, and surprising, differences. For example, the limit of the (normalized) discounted costs, as the discount vanishes, is not equal to the average cost, but rather equal to an arithmetic mean of the average cost over a range of values for the risk sensitivity coefficient. Note: The full version of this paper has been accepted for publication in the IEEE Transactions on Automatic Control.